Methods and apparatus to facilitate 3d object visualization and manipulation across multiple devices

ABSTRACT

Methods and apparatus to facilitate 3D object visualization and manipulation across multiple devices are disclosed. Example apparatus disclosed herein include a viewpoint determiner, a visible shard determiner, and a laminate assembler. The viewpoint determiner determines a viewpoint location of a viewpoint corresponding to a viewing device, the viewpoint location being in a reference frame of a three-dimensional (3D) model. The visible shard determiner determines a visible shard set of the 3D model based on the viewpoint location. The laminate assembler generates a two-dimensional (2D) image of the visible shard set.

PRIORITY CLAIM

This application is a continuation of, claims the benefit of andpriority to U.S. patent application Ser. No. 17/200,304, filed on Mar.12, 2021, which is a continuation of, claims the benefit of and priorityto U.S. patent application Ser. No. 16/257,393, filed on Jan. 25, 2019,now U.S. Pat. No. 10,950,044, which claims the benefit of and priorityto U.S. Provisional Patent Application No. 62/622,075, filed on Jan. 25,2018, each of which are incorporated herein by reference in theirentirety.

FIELD OF THE DISCLOSURE

This disclosure relates generally to three-dimensional (3D) computergraphics, and, more particularly, to methods and apparatus to facilitate3D object visualization and manipulation across multiple devices.

BACKGROUND

3D geometric models are often used to design products for manufacture(e.g., in architecture, construction, real estate, gaming, healthcare,scientific visualization, movies, etc.). During the design process,iterative changes to a 3D model are often made by electronically sendingan updated 3D model file back and forth between designers (e.g.,engineers, technicians, draftspeople, etc.). However, 3D model filesizes are often too large to be sent via conventional email and/orrequire long time periods to transfer from a sender to a recipient.Thus, current 3D design processes utilize specialized file transferprotocols (FTP) or shared local or hosted network storage betweendesigners and/or wait for extended periods for large 3D files totransfer.

SUMMARY

An example apparatus is disclosed. The example apparatus includes aviewpoint determiner, a visible shard determiner, and a laminateassembler. The viewpoint determiner determines a viewpoint location of aviewpoint corresponding to a viewing device, the viewpoint locationbeing in a reference frame of a 3D model in a virtual 3D environment.The visible shard determiner determines a visible shard set of the 3Dmodel based on the viewpoint location. The laminate assembler generatesa two-dimensional (2D) image based on the visible shard set.

An example method is disclosed. The example method includes determining,by executing an instruction with a processor, a viewpoint location of aviewpoint corresponding to a viewing device, the viewpoint locationbeing in a reference frame of a 3D model; determining, by executing aninstruction with the processor, a visible shard set of the 3D modelbased on the viewpoint location; and generating, by executing aninstruction with the processor, a 2D image based on the visible shardset.

An example tangible computer readable medium comprising exampleinstructions for execution by a processor is disclosed herein. Whenexecuted, the example instructions cause the processor to: determine aviewpoint location of a viewpoint corresponding to a viewing device, theviewpoint location being in a reference frame of a 3D model; determine avisible shard set of the 3D model based on the viewpoint location; andgenerate a 2D image based on the visible shard set.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example environment as disclosed herein.

FIG. 2 is a block diagram of an example model viewer used in the exampleenvironment of FIG. 1 .

FIG. 3 illustrates an example 3D model.

FIG. 4 illustrates the example 3D model of FIG. 3 represented astriangles.

FIG. 5 illustrates the example 3D model of FIGS. 3 and 4 represented astriangles grouped into shards.

FIG. 6 is an exploded view of the example 3D model of FIG. 5 .

FIG. 7 illustrates the example 3D model of FIGS. 3-5 occupying aplurality of voxels.

FIG. 8 illustrates the example 3D model viewed by a viewpoint via themodel viewer of FIG. 2 .

FIG. 9 illustrates a shard set of the example 3D model visible via theviewpoint of FIG. 8 , corresponding 2D tiles generated by the modelviewer of FIG. 2 , and a 2D laminate generated by the model viewer.

FIG. 10A illustrates example 2D tiles of the example 3D model of FIGS.3-6 generated by the example model viewer of FIG. 2 .

FIG. 10B illustrates example 2D tiles of the example 3D model of FIGS.3-6 generated by the example model viewer of FIG. 2 .

FIG. 11 is a side view of a ray corresponding to the viewpointintersecting one of the visible shards of FIG. 9

FIG. 12 illustrates a collection of 2D laminates of the example 3D modelgenerated by the example model viewer of FIG. 2 .

FIG. 13 is flowchart representative of machine readable instructionsthat may be executed to implement the example model viewer of FIG. 2 todisplay a 2D laminate.

FIG. 14 is a block diagram of an example computer platform capable ofexecuting the instructions of FIG. 13 to implement the example modelviewer of FIG. 2 .

The figures are not to scale. Wherever possible, the same referencenumbers will be used throughout the drawings and accompanying writtendescription to refer to the same or like parts.

DETAILED DESCRIPTION

FIG. 1 illustrates an example environment 110 as disclosed herein. Inthe illustrated examples of FIG. 1 , the environment 110 includes atleast one server 112, at least one viewing device 114, and a network116. In the illustrated example of FIG. 1 , the at least one viewingdevice 114 is a plurality of viewing devices 114 that includes a laptopcomputer 118, a tablet 120, a mobile device 122 (e.g., a smartphone),and a desktop computer 124. It should be understood that the viewingdevices 114 may include more or fewer viewing devices than the examplelaptop computer 118, the example tablet 120, the example mobile device122, and the example desktop computer 124 shown in FIG. 1 in anycombination of viewing devices types. In other words, the number ofviewing devices 114 included in the environment 110 is not limited tothe number and combination and viewing devices shown in FIG. 1 (e.g.,the laptop computer 118, the tablet 120, the mobile device 122, and thedesktop computer 124). The network 116 may be any type of electronicdata transfer system (e.g., the Internet, an intranet, a local areanetwork (LAN), etc.).

As shown in the illustrated example of FIG. 1 , the viewing devices 114are connected with the servers 112 via the network 116. The viewingdevices 114 may further be connected with one another via the network116. In the illustrated example of FIG. 1 , 3D model files 126 and/orportions of 3D model files 126 are stored on the servers 112. Inoperation, 2D renderings of the 3D model files are displayed on theviewing devices 114 via the network 116, as will be explained in greaterdetail below in conjunction with FIGS. 2-12 .

FIG. 2 is a block diagram of an example model viewer 202 used in theexample environment 110 of FIG. 1 . In the illustrated example of FIG. 2, the model viewer 202 includes a visible shard determiner 204, aplurality of tile generators 206 a to 206 n, a laminate assembler 208, aviewpoint determiner 210, a plurality of tile compositors 212 a to 212n, and an interface 214. It should be understood that the example tilegenerators 206 a to 206 n are substantially identical. It should befurther understood that the model viewer 202 may have any number of tilegenerators 206. It should be understood that the example tilecompositors 212 a to 212 n are substantially identical. It should befurther understood that the model viewer 202 may have any number of tilecompositors 206. In the illustrated example of FIG. 2 , the model vieweris in communication with a database 218. The visible shard determiner204, the tile generators 206 a to 206 n, the laminate assembler 208, thetile compositors 212 a to 212 n, and the database 218 may be stored inone or more of the servers 112 of FIG. 1 . In the illustrated example ofFIG. 2 , the viewpoint determiner 210 and the interface 214 may bestored in the one of the viewing devices 114 of FIG. 1 . Thus, the modelviewer 202 is stored on the servers 112 (e.g., a back end) and theviewing devices 114 (e.g., front ends).

In the illustrated example of FIG. 2 , the visible shard determiner 204is in communication with the tile generators 206 a to 206 n, with theviewpoint determiner 210, and with the database 218. The tile generators206 a to 206 n are further in communication with the laminate assembler208 and with the database 218. The tile compositors 212 a to 212 n arein communication with the laminate assembler 208. The interface 214 isin communication with the laminate assembler 208 and with the viewpointdeterminer 210.

In operation, the interface 214 receives display inputs from the viewingdevices 114. The display inputs include movement inputs (e.g., via amouse, keys, a touchscreen, a scroll wheel, etc.). The viewpointdeterminer 210 determines where to move viewpoints of the viewingdevices 114 in a reference frame about a 3D model based on the movementinputs, as will be explained in greater detail in conjunction with FIGS.3-8 . The visible shard determiner 204 accesses the database 218 todetermine which shards of a 3D model (e.g., the 3D model 126 of FIG. 1 )are visible based on the determined viewpoint, as will be explained ingreater detail in conjunction with FIGS. 3-8 . The tile generators 206 ato 206 n generate depth and color data of the determined visible shardsand render the visible shards into tiles (flattened 2D images), as willbe explained in greater detail in conjunction with FIGS. 9-11 . The tilegenerators 206 a to 206 n communicate the tiles and their respectivecolor and depth data to the laminate assembler 208. The laminateassembler 208 sends pairs of tiles and their respective color and depthdata to the tile compositors 212 a to 212 n. The tile compositors 212 ato 212 n compare the depth data of the tiles to generate a compositetile using color data selected based on the depth comparison, as will beexplained in greater detail in conjunction with FIGS. 9-11 . The tilecompositors 212 a to 212 n deliver the composite tiles to the laminateassembler 208. The laminate assembler 208 continues to send individualtiles and composite tiles to the tile compositors 212 a to 212 n untilall tiles originally generated by the tile generators 206 a to 206 n arecombined into a single 2D composite tile, referred to as a laminate, aswill be explained below in conjunction with FIGS. 9-11 . The laminateassembler 208 communicates the laminate image to the interface 214. Theinterface 214 displays the laminate on one of the of the viewing devices114.

FIG. 3 illustrates an example 3D model 302. FIG. 4 illustrates theexample 3D model 302 represented as triangles 410. FIGS. 5 and 6illustrate the example 3D model 302 represented as triangles 410 groupedinto first, second, third, fourth, fifth, sixth, seventh, and eighthshards 521, 522, 523, 524, 525, 526, 531, 532. FIG. 7 illustrates theexample 3D model 302 occupying a plurality of voxels 710. In theexamples of FIGS. 3-11 , the example 3D model 302 depicts a piston andconnecting rod assembly. In the illustrated examples of FIGS. 3-11 , the3D model 302 and the first, second, third, fourth, fifth, sixth,seventh, and eighth shards 521, 522, 523, 524, 525, 526, 531, 532 aredisposed in a reference frame 306. The example reference frame 306 isthree-dimensional Cartesian. It should be understood that the example 3Dmodel 302 may be disposed in any type of 3D reference frame (e.g.,cylindrical, spherical, etc.).

At a high level, examples disclosed herein may include taking a large 3Dmodel file (or many 3D models) and breaking the 3D model file intosmaller components referred to as shards that can be processedindependently to determine a 2D laminate image to be displayed to agiven user or viewing device. In practice, this may first includeprocessing the 3D model to generate a plurality of triangles or otherpolygons, which is sometimes referred to as tessellation. The examplesdisclosed herein may be described with reference to triangles. Howeverit should be understood that any object for which an intersection with aray may be determined can be used, including polygons, boundaryrepresentations (B-reps or BREPS), constructive solid geometry (CSG)trees, cones, etc. (e.g., model primitives). The triangles or othermodel primitives are then grouped into one or more shards. The trianglesmay be two dimensional (2D) with respect to the plane defined by theirvertices, or may be understood as 3D with respect to a reference frame,may have a set size or variable size, and when combined may form a meshor surface that follows the contours of the 3D model. In some examplesthere may be hundreds, thousands, millions, or more triangles thatrepresent a given model. The number and density of triangles may bechanged to allow for greater or lesser resolution of the 3D model. Forinstance, a simple 3D model may have fewer triangles than a more complexmodel, to allow for greater definition of the contours of the complexmodel. Further, the complex model may include a greater density oftriangles to account for relatively small features of the model. In someexamples, the number and density of triangles may vary across the model,such that more complex areas are represented by a greater number and/orgreater density of triangles.

Triangles may be grouped together and combined into shards (e.g., thefirst, second, third, fourth, fifth, sixth, seventh, and eighth shards521, 522, 523, 524, 525, 526, 531, 532). The number of triangles pershard may vary, and in some cases there may be an upper limit such astwenty thousand triangles per shard. Other numbers, limits, and rangesare possible as well. The number of triangles per shard may affect oneor more processing benchmarks or metrics, and as such may be changed ordynamically modified to maintain a particular frame rate or processingspeed, for example. Further, the density of the triangles may affect thesize and shape of each shard.

To determine which triangles are grouped into a given shard, one or moreprocessing techniques may be used. In some examples, the triangles maybe grouped based on a particular component of the 3D model. Forinstance, a 3D model may have multiple components (e.g., a cylinder witha handle may comprise two components—the cylinder and the handle), andall the triangles corresponding to a given component may be grouped intoa shard. Alternatively, one or more components may include a greaternumber of triangles than can be grouped into a single shard, and as sucha component may be represented by multiple shards. This may occur with acomponent is large, complex, or is shown in high resolution or detail.Further, a single shard represent multiple components. Thus, a singleshard may include triangles that represent a single component, part of acomponent, or multiple components.

There are multiple techniques for grouping the triangles into shards. Inone example, adjacent triangles may be grouped together to form a shard.This technique may be based on triangle adjacency. In another example,edge walking may be used to determine groupings, wherein a firsttriangle is selected and additional triangles are selected by movingalong an edge to a next triangle. In general, techniques such asnearest-neighbor grouping or hierarchical clustering may be used. Thesetechniques may involve organizing the triangles or other geometricprimitives into a hierarchical structure that describes the proximity ofeach triangle to one or more other triangles. The hierarchical structuremay be analyzed and used to group the triangles into clusters based onone or more features, which may in turn result in a grouping of a givennumber of triangles into a shard. Further, in some examples trianglesmay be grouped based on the component they represent, or other metadataassociated with the triangles. Once the shards are determined, they maybe stored in a database for later use. FIGS. 5 and 6 show that thenumber, size, density, and other characteristics of the triangles 410may differ within a shard and across shards of the represented object.

In some examples, a 3D model may be broken into triangles and shardswhich may be stored on a single device, across multiple devices, and/orredundantly across multiple devices.

In order to display a laminate or 2D image of a given object to a user,example embodiments may include receiving information from the userdevice (e.g., a viewpoint, viewing position, or orientation) which maybe used to determine or generate a laminate. FIG. 8 shows an exampleviewpoint position 804. This process may include several steps, whichmay include (i) determining a viewpoint position/orientation withrespect to the 3D model, (ii) determining which shard(s) are visiblefrom the determined position, (iii) generating a plurality of processingtasks for generating 2D images of the visible shard(s), (iv) combiningthe generated 2D images into a single image to be presented to theviewing device. These steps are discussed in further detailed below.

First, a position, orientation, or other spatial orientation data of theviewing device requesting the laminate may be determined. Thisinformation may be transmitted from the viewing device to the modelviewer or processing device via the interface. Where the system includesmultiple users viewing a given model on multiple devices, an ID of aparticular viewing device or user account may be used to identify theuser, determine one or more permissions or security actions available,or to allow one or more available features (e.g., bookmarking, notetaking, object or component manipulation, etc.). The position andorientation of the requesting viewing device may correspond to an angleat which the viewing device “sees” the object. This is described belowwith respect to FIG. 8 .

Based on a determined position and orientation, the visible shards ofthe object may be determined. Ordinarily, all shards of the 3D model maybe selected as visible and processed to determine the 2D image displayedto the user. However, one or more culling techniques may be used toreduce the number of shards deemed as visible and therefore the amountof processing power needed. This can be done in several ways, includingfrustum culling, occlusion culling, and level of detail culling, forexample.

Frustum culling may remove from consideration any shards that arecompletely outside a view frustum corresponding to the viewpoint of theviewing device. FIG. 8 illustrates an example view frustum 812. As such,when a user zooms in or can only see a subset of the total 3D model,shards corresponding to parts outside the view frustum may not be usedto produce the 2D image of the model. In some examples, a given shardmay be partially in and partially outside the view frustum. In thesecases, any shard that is even partially in the view frustum may beconsidered, and only those shards that are completely outside the viewfrustum may be removed from consideration. In the example of FIG. 8 ,the seventh and eighth shards 531, 532 are completely outside the viewfrustum 812 and are thus excluded from consideration. In contrast, inthe example of FIG. 8 , the fifth shard 525 is partially present in theview frustum 812 and will thus be considered.

Occlusion culling may include determining one or more shards that areeither partially or completely blocked by another shard, with respect tothe determined viewpoint. For example, FIGS. 5, 6, and 8 show that thesixth shard 526 is blocked from view by viewpoint 804 by the first,second and fourth shards 521, 522, 524. As such, the sixth shard 526 maybe removed from consideration, and not processed to determine the 2Dimage transmitted to the viewing device.

Level of detail culling may include determining whether a given shardshould be removed from consideration based on the level of detail of theobject shown. For instance, where the viewpoint is far away from theobject, a given shard may take up one pixel or a small number of pixels.In this case, the shard may be removed from consideration when doing sowould not greatly affect the image. The resolution and viewpointposition/orientation may cause one or more shards to be removed fromconsideration and to not be processed.

Once the list of shards is determined based on the culling techniqueslisted above, and one or more other processing techniques, a pluralityof processing tasks may be determined. The processing tasks may eachcorrespond to a visible shard or plurality of visible shards, and mayinclude rendering the shard or shards. Put another way, once the list ofvisible shards is determined the shards may be independently processedto determine a plurality of 2D rendered images, which may then becombined and sent to the viewing device.

Each of the plurality of tasks may be a rendering task, which may beperformed by one or more computing devices. In some examples, one ormore subscriber devices may be used to process the tasks in series on afirst come first served basis. For instance, where there are 10 shardsvisible, there may be 10 rendering tasks which must be completed todetermine the 2D image to be presented to the viewing device. In someexamples, there may be a single subscriber device that completes all therendering tasks. There may also be two or more subscriber devices, whichmay process the 10 rendering tasks in parallel. For instance, the firstsubscriber may take the first rendering task while the second subscribertakes the second rendering task. The first subscriber may include a morepowerful processor than the second subscriber, and as such may completethe first rendering task quickly. The first subscriber may then take andcomplete the third rendering task, and then take and complete the fourthrendering task before the second subscriber has completed the secondrendering task. As such, rendering tasks may be completed in any order,in parallel, and by one or more devices. In this manner, a plurality ofsubscriber devices may complete the rendering tasks in an efficientmanner such that the subscriber with the fastest processor may completemore rendering tasks than a slower subscriber.

Each processing task may include rendering a visible shard. In someexamples, this may include performing a 3D rendering operation based onthe determined viewpoint location, such as ray tracing, rasterization,or other such operations that project 3D geometric shapes onto twodimensions. Examples disclosed herein may be described with respect toray tracing in particular, however it should be noted that any otheroperation for projecting 3D geometric shapes on to two dimensions may beused. A subscriber completing the rendering task may receive anidentifier of a shard that must be rendered, and responsively retrievethe triangles that make up the shard from the database. The subscribermay then perform a ray tracing operation, to determine a color and depthfor each pixel. The ray tracing operation may be understood as sendingout a ray starting at the viewpoint location, passing through each pixelthat will be displayed. The ray may pass through the pixel and contact atriangle of the shard being rendered. The ray may then bounce off thetriangle and contact one or more other triangles or light sources. Thecolor of a given pixel may correspond a component to which the trianglebelongs. Each pixel may also include shading and lighting information,which may correspond to a depth of the triangle that corresponds to thepixel. As such, each rendered pixel may comprise both color and depthinformation. The depth of a given pixel may be determined as aone-dimensional distance along the ray to the intersection of the rayand the intersected triangle, or other 3D geometric primitive (cube,cone, polygons, B-reps, CSG trees, etc.). Ray tracing is furtherexplained below in conjunction with FIG. 11 .

In some examples, a derivative 3D rendering acceleration data structuresuch as a boundary volume hierarchy (BVH), voxel hierarchy, spatialgrid, etc. may be determined for each shard. Examples disclosed hereinmay be described with respect to a BVH, but it should be noted thatother data structures can be used as well. The BVH may be stored andaccessed by one or more subscribers, such that when a given shard mustbe rendered it can be rendered more quickly. This may be particularlyuseful when a viewpoint changes and a shard must be re-rendered, and anew subscriber is tasked with rendering the shard. In this case, the BVHmay be retrieved by the subscriber from the database, and used to morequickly render the shard.

3D rendering acceleration data structures such as a BVH may reduce thenumber of ray-triangle (or other geometric primitive) intersection teststhat must be performed against a single ray in a ray tracing or scanline rasterization. This reduction is achieved by grouping triangles (orother geometric primitives) spatially and representing the group withanother, single geometric primitive. As the surrogate primitives arecreated they may be recursively, progressively grouped with otherprimitives into a hierarchical tree structure. Intersections performedtop-down against this tree structure effectively and efficiently enableculling of entire branches of the tree. This eliminates substantialnumbers of leaf geometric primitives against which intersection testsmust be performed.

Voxels (e.g., the plurality of voxels 710) may be used in one or moreculling operations in order to determine the minimum plurality of shardprocessing jobs or rendering jobs required to produce a single laminate.As shown in FIG. 7 , the shards 521-526, 531, 532 occupy at least onevoxel 710. In other words, each of the shards 521-526, 531, 532corresponds to at least one three-dimensional quantized block of spaceof the reference frame 306—one of the voxels 710. Thus, whether a voxel710 is present in the view frustum 812, whether the voxel 710 contains ashard, and which shard the voxel 710 contains may be determined by themodel viewer 202, as will be explained in greater detail below inconjunction with FIG. 8 .

In some examples, rasterization or another rendering technique may beused instead of ray tracing. Then, regardless of the technique used, theprocessing task (i.e., rendering task) may result in a 2D imagecomprising pixels having color and depth information corresponding to agiven shard. Where multiple shards are visible, multiple 2D images maybe determined. The multiple 2D images may then be combined into a single2D image comprising the plurality of visible shards. This combined 2Dimage may then be transmitted to the viewing device.

FIG. 8 illustrates the example 3D model 302 viewed by the viewpoint 804via the model viewer 202 of FIG. 2 . In the illustrated example of FIG.8 , the viewpoint 804 is depicted as a motion-picture film camera. Theviewpoint 804 is associated with one of the viewing devices 114 of FIG.1 . It should be understood that each of the viewing devices 114 isassociated with a respective viewpoint and that multiple viewpoints mayview the 3D model 302 simultaneously. As shown in FIG. 8 , the viewpoint804 views the 3D model through the view frustum 812. In other words, theviewpoint 804 sees the portions of the 3D model 302 that are included inthe view frustum 812. Thus, as shown in FIG. 8 , portions of the 3Dmodel 302 outside of the view frustum 812 are hidden from the viewpoint804. In operation, the 3D model 302 remains stationary in the referenceframe 306 while the viewpoint 804 moves about, translates, approaches(e.g., zooms in), and retreats (e.g., zooms out) in the reference frame306 with respect to the 3D model 302, as indicated by the double arrows808 and arrowed arcs 810. As an analogy, the 3D model acts as a fixedstatue and the viewpoint 804 is an observer walking around the statue tolook at all sides of the statute. In operation, 2D assembled laminateimages of the 3D model 302 based on the respective positions of theviewpoints observing the 3D model (including viewpoint 804) are renderedand communicated by the servers 112 of FIG. 1 for display on the viewingdevices 114 of FIG. 1 . In other words, what the viewpoints (e.g.,viewpoint 804) sees is shown as 2D laminates on the viewing devices 114.In yet other words, users view 2D laminates of the 3D model 302 from thevantage points of the viewpoints via the viewing devices 114. It shouldbe understood and appreciated that generating and communicating 2Dlaminates of the 3D model 302 is more efficient than transmitting allthe data that makes up the 3D model and rendering/re-rendering the 3Dmodel for different viewpoints. In other words, it is easier to storeall the 3D model data at one or more server locations (e.g., in thecloud) and send simple 2D laminate images to the viewing devices 114than to transmit the 3D model data. Thus, data transmission efficiencyis improved by reducing the amount of data transferred to the viewingdevices 114. In operation, the viewpoint determiner 210 of FIG. 2determines the positions of the viewpoints in the reference frame 306.

As shown in FIG. 8 , voxels 710 containing the first, second, third,fourth, fifth, and sixth shards 521, 522, 523, 524, 525, 526 aredisposed in the view frustum 812 and are thus visible to the viewpoint804. Voxels 710 containing the seventh and eighth shards 531, 532 areoutside of the view frustum 812 and are thus hidden from the viewpoint812. Thus, the first, second, third, fourth, fifth, and sixth shards521, 522, 523, 524, 525, 526 make up a visible shard set 820. It shouldbe understood and appreciated that despite being only partially includedin the view frustum 812, as shown in FIG. 8 , the fifth shard 525 isincluded in the visible shard set 820. Looking at FIGS. 6, 7, and 8 , itshould be further understood and appreciated that despite being obscuredby the first shard 521, the sixth shard 526 is present in the viewfrustum 812 and is thus included in the visible shard set 820. Further,because the seventh and eighth shards 531, 532 are in voxels 710completely outside the view frustum 812, the seventh and eighth shards531, 532 make up a hidden shard set 830. In other words, from thevantage point of the viewpoint 804, the visible shard set 820 is visiblewhile the hidden shard set 830 is hidden from view.

In operation, using the examples of FIGS. 3-7 the visible sharddeterminer 204 makes a visible shard determination that the first,second, third, fourth, fifth, and sixth shards 521, 522, 523, 524, 525,526 are visible to the viewpoint 804 based on the position of theviewpoint 804 in the reference frame 306. In other words, for theexamples of FIGS. 3-7 , the visible shard determiner 204 determines thata set of visible shards—the visible shard set 820—for the viewpoint 804includes the first, second, third, fourth, fifth, and sixth shards 521,522, 523, 524, 525, 526. Further in operation, the visible sharddeterminer 204 communicates the visible shard set 820 to the tilegenerators 206 a to 206 n. It should be appreciated that the visibleshard determination made by the visible shard determiner 204 isquantized. In other words, even if only a portion of a shard is visiblefor a given viewpoint or the shard is obscured, the visible sharddeterminer 204 includes the partially-visible and/or obscured shard inthe set of visible shards, as described above with respect to the fifthand sixth shards 525, 526.

FIG. 9 illustrates the visible shard set 820 of the example 3D model302, corresponding 2D tiles generated by the model viewer 202, and a 2Dlaminate 930 generated by the model viewer 202. FIGS. 10A and 10Billustrate example 2D tiles of the example 3D model 302 generated by theexample model viewer 202. FIG. 11 is a side view of a ray 1120corresponding to the viewpoint 804 intersecting the first shard 521,which is part of the visible shard set 820.

In operation, the tile generators 206 a to 206 n generate 2D tiles forthe viewpoint 804 based on the visible shard set 820 shown in FIG. 8 .More specifically, one or more of the tile generators 206 a to 206 nretrieve the first, second, third, fourth, fifth, and sixth shards 521,522, 523, 524, 525, 526, the from the database 218 of FIG. 2 . In otherwords, individual shards of the set of visible shards 820 may bedistributed amongst multiple tile generators 206. The tile generators206 a to 206 n respectively render (flatten) the 3D the first, second,third, fourth, fifth, and sixth shards 521, 522, 523, 524, 525, 526,into 2D tiles. As shown in FIGS. 9, 10A, and 10B, the first shard 521 isflattened in a 2D first tile 921. The second shard 522 is flattened intoa 2D second tile 922. The third shard 523 is flattened into a 2D thirdtile 923. The fourth shard 524 is flattened into a 2D fourth tile 924.The fifth shard 525 is flattened into a 2D fifth tile 925. The sixthshard 526 is flattened into a 2D sixth tile 926. As shown in FIGS. 10Aand 10B, the first, second, third, fourth, fifth, and sixth tiles 921,922, 923, 924, 925, 926 are in a 2D reference frame 906. Thus, theentire set of visible shards 820 is flattened into 2D tiles by the tilegenerators 206 a to 206 n.

In the illustrated example of FIG. 11 , the ray 1120 extendsperpendicularly from a pixel 1140 of a viewing plane 1110. The viewingplane 1110 is the 2D area visible by the viewpoint 804 in the viewfrustum 812. It should be understood that the viewing plane 1110includes a plurality of pixels such as the pixel 1140 arranged incolumns and rows. As shown in FIG. 11 , the first shard 521 has a colorvalue c (e.g., red, blue, green, gray, or any color on the visiblespectrum) and a depth value d at the intersection point 1130 where theray 1120 intersects the first shard 521. This color and depth data iscollectively shown as (c, d) in FIG. 11 . The depth value d is adistance from the viewing plane 1110 to the intersection point 1130. Thecolor value c is the color of the first shard 521 at the intersectionpoint 1130 as viewed by the viewpoint 804 via the viewing plane 1110 atthe pixel 1140. In other words, as the viewpoint 804 looks at theviewing plane 1110, pixel 1140 has color value c and is associated withdepth value d.

In operation, the tile generators 206 a to 206 n determine color anddepth data for individual visible shards for every point where a rayintersects the visible shard. Thus, for each individual visible shard,color and depth data is generated for each pixel of the viewing planethrough which the individual shard is seen. The color data from eachpixel forms a 2D tile and the depth data is associated with each pixel(e.g., as metadata). In other words, a 2D tile is a collection ofcolored pixels and depth data is associated with each of the coloredpixels. The tile generators 206 a to 206 n provide the tiles andassociated depth data to the laminate assembler 208. In some examples,the tile generators 206 a to 206 n may further determine transparencydata for visible shard ray intersection points.

In operation, the laminate assembler 208 sends pairs of tiles (e.g.,first tile 921 with sixth tile 926, third tile 923 with fourth tile 924,etc.) and associated depth and transparency data to the tile compositors212 a to 212 n to be merged into composite tiles. Thus, the work ofcompositing the pairs of tiles is spread amongst the tile compositors212 a to 212 n.

In operation, the tile compositors 212 a to 212 n compare the respectivedepth data of each pixel of the tiles provided by the laminate assembler208. For a given pixel location in the tiles, the tile compositors 212 ato 212 n select the colored pixel between the pair of tiles having thesmallest depth value. In other words, a tile compositor 212 merges apair of tiles into a composite tile based on which pixels between thepair of tiles were closest to the viewing plane 1110. Thus, eachcomposite tile is a collection of the colored pixels between the pair oftiles that were nearest to the viewing plane 1110. Further, each pixelof the composite tile thus has color data and depth data. The tilecompositors 212 a to 212 n then deliver the composite tiles to thelaminate assembler 208. In examples where the tiles include transparencydata, the tile compositors 212 a to 212 n adjust the color data of theselected pixel based on the transparency data and the color data of thenon-selected pixel.

Further in operation, the laminate assembler 208 continues the tilecompositing process by sending pairs of composite tiles and originaltiles or pairs of composite tiles to the tile compositors 212 a to 212 nuntil all of the tiles originally delivered by the tiles generators 206a to 206 n are composited into a final laminate 930. In the example ofFIG. 9 , the 2D laminate 930 is a composited 2D image of the 3D model302 as observed by the viewpoint 804 in the view frustum 812.

Further, the laminate assembler 208 sends the 2D laminate 930 to theviewing device 114 for display on the viewing device 114. In otherwords, as shown in FIG. 9 , the tile generators 206 a to 206 n convertthe visible shard set 820 into 2D tiles 921-926, the tile compositors212 a to 212 n successively composite the tiles 921-926 according topixel depth data into the laminate 930, and the laminate assembler 208delivers the laminate 930 to the viewing device 114.

FIG. 12 illustrates a collection of 2D laminates 1212 of the example 3Dmodel 302 generated by the example model viewer 202 of FIG. 2 . In theillustrated example of FIG. 12 , the collection of 2D laminates 1212includes the first laminate 930 of FIG. 9 , a second laminate 932, athird laminate 933, a fourth laminate 934, and a fifth laminate 935.Each of the first, second, third, fourth, and fifth laminates 930, 932,933, 934, 935 respectively correspond to different viewpoint positionsin a reference frame (e.g., the reference frame 306 of FIGS. 3-11 )about the example 3D model 302. In other words, the first, second,third, fourth, and fifth laminates 930, 932, 933, 934, 935 are images ofwhat is observed by the viewpoint 804 of the 3D model 302 at first,second, third, fourth, and fifth positions in the reference frame 306.

In operation, for the example of FIG. 12 , the tile generators 206 a to206 n, the tile compositors 212 a to 212 n, and the laminate assembler208 of FIG. 2 successively generate the first, second, third, fourth,and fifth laminates 930, 932, 933, 934, 935 as the viewpoint 804 ismoved in the reference frame 306. In operation, the tile generators 206a to 206 n generate 2D tiles and the tile compositors 212 a to 212 ncomposite 2D laminates as the viewpoint 804 is moved between viewpointpositions according to a frame rate (e.g., 24 frames per second, 40frames per second, etc.). The tile generators 206 a to 206 n do notgenerate new 2D tiles when the viewpoint remains fixed at a viewpointlocation in virtual 3D space, thus conserving processing capacity. Inother words, the tile generators 206 a to 206 n generate 2D tiles forcomposition into 2D laminates by the tile compositors 212 a to 212 n inresponse to movement inputs from a viewing device 114.

Further in operation, the interface 214 of FIG. 2 displays therapidly-generated laminates at a viewing device (e.g., one of theviewing devices 114 of FIG. 1 ). Thus, an illusion of motion (e.g.,rotation, translation, etc.) of the 3D model is produced for a user ofthe viewing device by rapidly displaying the static laminates on theviewing device. In other words, the model viewer 202 creates anelectronic flipbook of static laminates that are rapidly shown to a uservia the viewing device 114. In the illustrated example of FIG. 12 , the3D model 302 thus appears to rotate in the collection of laminates 1212.Returning to the statue analogy, the model viewer 202 flips a stack of2D images taken about the fixed statue in front of a user. Although theuser may be standing still, the statue appears to rotate in front of theuser. Thus, in the illustrated example of FIG. 12 , the 3D model 302appears to rotate as the first, second, third, fourth, and fifthlaminates 930, 932, 933, 934, 935 progress from left to right.

Embodiments included herein have been described involving the display ofvarious images, laminates, shards, and other visual features on a screenor display device. It should be appreciated that certain examples mayinclude the use of the concepts described herein in the context ofaugmented reality and/or virtual reality applications. In an augmentedreality context, images, laminates, shards, or other visual features maybe provided along with a transparent background. The transparentbackground may allow a display device to superimpose the provided imageonto a local camera image or view, such that the object in the image,laminates, or shards appears to be located within the camera field ofview. This is one example of how the disclosed concepts can be used withaugmented reality. It should be appreciated that other techniques may beused as well. Additionally, the concepts disclosed herein may be used ina virtual reality context. For example, multiple images, laminates, ordisplayed shards may be provided to a device in a stereographic mannerto enable the device to be used for a virtual reality display. Providingdual images for a stereographic display using the techniques andconcepts disclosed herein can remove the need for significant processingat the display device itself. In this way, the virtual reality displaydevice can be more compact, power efficient, and of a simpler and morecost-effective design.

FIG. 13 is flowchart representative of machine readable instructionsthat may be executed to implement the example model viewer 202 of FIG. 2to display a laminate. The flowchart of FIG. 13 is representative ofmachine readable instructions that are stored in memory (such as thememory 1404 of FIG. 14 ) and include one or more methods which, whenexecuted by a processor (such as the processor 1406 of FIG. 14 ), causethe environment 110 to implement the example model viewer 202 of FIG. 2. While example program 1300 is described with reference to theflowchart illustrated in FIG. 13 , many other methods of implementingthe example model viewer 202 may alternatively be used. For example, theorder of execution of the blocks may be rearranged, changed, eliminated,and/or combined to perform the methods 1300. Further, because the method1300 is disclosed in connection with the components of FIG. 2 , somefunctions of those components will not be described in detail below.

The terms “non-transitory computer-readable medium” and“computer-readable medium” include a single medium or multiple media,such as a centralized or distributed database, and/or associated cachesand servers that store one or more sets of instructions. Further, theterms “non-transitory computer-readable medium” and “computer-readablemedium” include any tangible medium that is capable of storing, encodingor carrying a set of instructions for execution by a processor or thatcause a system to perform any one or more of the methods or operationsdisclosed herein. As used herein, the term “computer readable medium” isexpressly defined to include any type of computer readable storagedevice and/or storage disk and to exclude propagating signals.

In the illustrated example of FIG. 13 , the example interface 214receives an input (e.g., a movement input, a laminate display request,etc.) from a viewing device 114 (block 1302). The viewpoint determiner210 determines a viewpoint location in a reference frame associated witha 3D model corresponding to the input (block 1304). The visible sharddeterminer 204 accesses the database 218 to determine a visible shardset of the 3D model based on the viewpoint location (block 1306). Thevisible shard determiner 204 assigns individual shards of the visibleshard set to the tile generators 206 a to 206 n (block 1308). The tilegenerators 206 a to 206 n retrieve the assigned visible shards from thedatabase 218 (block 1310). The tile generators 206 a to 206 n render theassigned visible shards into 2D tiles and deliver the 2D tiles to thelaminate assembler 208 (block 1312). The tile compositors 212 a to 212 nsuccessively merge 2D tiles received from the laminate assembler 208 toassemble the 2D tiles into a 2D laminate (block 1314). The interface 214sends the 2D laminate to the viewing device 114 (block 1316). The method1300 then ends.

FIG. 14 is a block diagram of an example computing platform 1402 capableof executing the instructions of FIG. 13 to implement the example modelviewer 202 of FIG. 2 . In the illustrated example of FIG. 14 , thecomputing platform 1402 includes a memory 1404, a processor 1406, inputdevice(s) 1408, an interface 1410, output device(s) 1412, and a bus1414.

In the illustrated example of FIG. 14 , the memory 1404, the processor1406, the interface 1410, and the output device(s) 1412 are incommunication with one another via the bus 1414. The input device(s)1408 are in communication with the interface 1410.

In the illustrated example of FIG. 14 , the processor 1406 of theon-board computing platform 1402 is structured to include the modelviewer 202. The processor 1406 may be any suitable processing device orset of processing devices such as, but not limited to, a microprocessor,a microcontroller-based platform, an integrated circuit, one or morefield programmable gate arrays (FPGAs), and/or one or moreapplication-specific integrated circuits (ASICs). The memory 1404 may bevolatile memory (e.g., RAM including non-volatile RAM, magnetic RAM,ferroelectric RAM, etc.), non-volatile memory (e.g., disk memory, FLASHmemory, EPROMs, EEPROMs, memristor-based non-volatile solid-statememory, etc.), unalterable memory (e.g., EPROMs), read-only memory,and/or high-capacity storage devices (e.g., hard drives, solid statedrives, etc). In some examples, the memory 1404 includes multiple kindsof memory, particularly volatile memory and non-volatile memory.

The memory 1404 is computer readable media on which one or more sets ofinstructions 1418, such as the software for operating the methods of thepresent disclosure, can be embedded. The instructions 1418 may embodyone or more of the methods or logic as described herein. For example,the instructions 1418 reside completely, or at least partially, withinany one or more of the memory 1404, the computer readable medium, and/orwithin the processor 1406 during execution of the instructions 1418.

The interface 1410 may be implemented by any type of interface standard(e.g., Ethernet, universal serial bus (USB), and/or a peripheralcomponent interconnect (PCI) express interface). The interface 1410includes a communication device (e.g., a transmitter, a receiver, atransceiver, a modem, network interface card, etc.) to exchange datawith external machines and/or computing devices via a network 1416(e.g., an Ethernet connection, wireless connection, a telephone line,coaxial cable, a cellular telephone system, etc.).

The machine readable instructions 1418 may be stored in the memory 1404and/or on a removable tangible computer readable medium storage (e.g., acompact disc, a digital versatile disc, a Blu-ray disc, a USB drive,etc.).

In the illustrated example of FIG. 14 , the input device(s) 1408 enablea user, such as an operator or technician, to provide instructions,commands, and/or data to the processor 1406. Examples of the inputdevice(s) 1408 include one or more of a button, a control knob, aninstrument panel, a touch screen, a touchpad, a keyboard, a mouse, aspeech recognition system, etc.

The output device(s) 1412 of the illustrated example display outputinformation and/or data of the processor 1406 to a user, such as anoperator or technician. Examples of the output device(s) 1412 include aliquid crystal display (LCD), an organic light emitting diode (OLED)display, a flat panel display, a touch screen, a solid state display,and/or any other device that visually presents information to a user.Additionally or alternatively, the output device(s) may include one ormore speakers and/or any other device(s) that provide audio signals fora user. Further, the output device(s) 1412 may provide other types ofoutput information, such as haptic signals.

From the foregoing, it will be appreciated that the above disclosedmethods and apparatus may aid in simultaneous viewing of 3D modelsduring a product design process. Thus, users may collaborate with oneanother in real time to make comments regarding the product representedby the 3D model. Additionally, the above disclosed methods and apparatusprovide a specific improvement to computer-related technology byreducing the number of times large 3D model files are transferred viaemail, FTP, etc. during a design process, thus freeing a processor toperform other tasks more quickly and consuming less energy.

Although certain example methods, apparatus, and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus, and articles of manufacture fairly falling within the scopeof the claims of this patent.

What is claimed is:
 1. An apparatus comprising: a viewpoint determinerto determine a viewpoint location of a viewpoint corresponding to aviewing device, the viewpoint location being in a reference frame of athree-dimensional model; a visible shard determiner to determine avisible shard set of the three-dimensional model based on the viewpointlocation; and a tile compositor to generate a two-dimensional imageassociated with the reference frame of the determined viewpoint locationof the three-dimensional model, wherein: the generated two-dimensionalimage is based on the visible shard set of the three-dimensional model,and the three-dimensional model existed prior to the generation of thetwo-dimensional image.
 2. The apparatus of claim 1, wherein the visibleshard set is retrieved from a database.
 3. The apparatus of claim 1,wherein the visible shard determiner communicates the visible shard setto the tile compositor.
 4. The apparatus of claim 1, wherein thegenerated two-dimensional image is based on transparency data associatedwith a visible shard ray intersection point.
 5. The apparatus of claim1, wherein an individual visible shard of the visible shard set is basedon transparency data, color data and depth data associated with thatindividual visible shard.
 6. A system comprising: a processor; and amemory device that stores a plurality of instructions that, whenexecuted by the processor, cause the processor to: determine a viewpointlocation of a viewpoint corresponding to a viewing device, the viewpointlocation being in a reference frame of a three-dimensional model;determine a visible shard set of the three-dimensional model based onthe viewpoint location; and generate a two-dimensional image of thevisible shard set associated with the reference frame of the determinedviewpoint location of the three-dimensional model, wherein thethree-dimensional model existed prior to the generation of thetwo-dimensional image.
 7. The system of claim 6, wherein the memorydevice stores a plurality of further instructions, that when executed bythe processor, cause the processor to retrieve the visible shard setfrom a database.
 8. The system of claim 6, wherein the memory devicestores a plurality of further instructions, that when executed by theprocessor, cause the processor to assign individual shards of thevisible shard set.
 9. The system of claim 6, wherein the memory devicestores a plurality of further instructions, that when executed by theprocessor, cause the processor to generated the two-dimensional imagebased on transparency data associated with a visible shard rayintersection point.
 10. The system of claim 6, wherein an individualvisible shard of the visible shard set is based on transparency data,color data and depth data associated with that individual visible shard.11. A method comprising: determining, by a processor, a viewpointlocation of a viewpoint corresponding to a viewing device, the viewpointlocation being in a reference frame of a three-dimensional model;determining, by the processor, a visible shard set of thethree-dimensional model based on the viewpoint location; and generating,by the processor, a two-dimensional image associated with the referenceframe of the determined viewpoint location of the three-dimensionalmodel, wherein the generated two-dimensional image is based on thevisible shard set of the three-dimensional model, and thethree-dimensional model existed prior to the generation of thetwo-dimensional image.
 12. The method of claim 11, wherein generatingthe two-dimensional image includes retrieving the visible shard set froma database.
 13. The method of claim 11, further comprising assigningindividual shards of the visible shard set.
 14. The method of claim 11,wherein the generated two-dimensional image is based on transparencydata associated with a visible shard ray intersection point.
 15. Themethod of claim 11, wherein an individual visible shard of the visibleshard set is based on transparency data, color data and depth dataassociated with that individual visible shard.